Organizing a Repo

Let’s go back to my ‘template’ for organizing an R repo.

├── _targets    <- stores the metadata and objects of your pipeline
├── renv        <- information relating to your R packages and dependencies
├── data        <- data sources used as an input into the pipeline
├── src         <- functions used in project/targets pipeline
|   ├── data    <- functions relating to loading and cleaning data
|   ├── models    <- functions involved with training models
|   ├── reports   <- functions used in generating tables and visualizations for reports
├── _targets.R    <- script that runs the targets pipeline
├── renv.lock     <- lockfile detailing project requirements and dependencies

The Problem

A predictive modeling workflow typically consists of a number of interconnected steps.

flowchart LR
raw[Raw Data] --> clean[Clean Data]
clean --> train[Training Set]
clean --> valid[Validation Set]
train --> preprocessor(Preprocessor)
preprocessor --> resamples[Bootstraps]
resamples --> model(glmnet)
model --> features(Impute + Normalize)
features --> tuning(Tuning)
tuning --> valid
preprocessor --> valid
valid --> evaluation[Model Evaluation]
train --> final(Model)
valid --> final

The Problem

Your project might end up looking something like this:

• 01-load.R
• 02-tidy.R
• 03-model.R
• 04-evaluate.R
• 05-deploy.R
• 06-report.R

The Problem

How do you insert a change into this process? Like, if you make a change to a function, how do you know what needs to be re run?

The Problem

This pattern of developing, changing, re-running can consume a lot of time, especially with time-consuming tasks like training models.

The Problem

It might not be too bad when you’re actively working on a project, but suppose you’re coming back to something after a few months away.

what is targets

Data analysis can be slow. A round of scientific computation can take several minutes, hours, or even days to complete. After it finishes, if you update your code or data, your hard-earned results may no longer be valid. Unchecked, this invalidation creates a chronic Sisyphean loop

The solution to this problem is to develop pipelines, which track dependencies between steps, or “targets”, of a workflow.

Most pipeline tools, such as Make, are either language agnostic or depend on using Python.

├── _targets    <- stores the metadata and objects of your pipeline
├── renv        <- information relating to your R packages and dependencies
├── data        <- data sources used as an input into the pipeline
├── src         <- functions used in project/targets pipeline
|   ├── data   <- functions relating to loading and cleaning data
├── _targets.R    <- script that runs the targets pipeline
├── renv.lock     <- lockfile detailing project requirements and dependencies

targets adds two main pieces to a project:

What is targets.R?

Running targets::use_targets() will create a template for the targets.R script, which all follow a similar structure.

# Created by use_targets().
# Follow the comments below to fill in this target script.
# Then follow the manual to check and run the pipeline:
#   https://books.ropensci.org/targets/walkthrough.html#inspect-the-pipeline

# Load packages required to define the pipeline:
library(targets)
# library(tarchetypes) # Load other packages as needed.

# Set target options:
tar_option_set(
  packages = c("tibble") # Packages that your targets need for their tasks.
  # format = "qs", # Optionally set the default storage format. qs is fast.
)

# Run the R scripts in the R/ folder with your custom functions:
tar_source()
# tar_source("other_functions.R") # Source other scripts as needed.

# Replace the target list below with your own:
list(
  tar_target(
    name = data,
    command = tibble(x = rnorm(100), y = rnorm(100))
    # format = "qs" # Efficient storage for general data objects.
  ),
  tar_target(
    name = model,
    command = coefficients(lm(y ~ x, data = data))
  )
)

An Example - Star Wars

Going back to our Star Wars sentiment analysis, we can build a simple targets pipeline to recreate what we did earlier. The basic steps of our pipeline will look something like this:

This is what the resulting pipeline will look like:

library(targets)

# set options
tar_option_set(packages = c("readr", "dplyr", "sentimentr", "here"))

# functions to be used

# load starwars data
load_data = function(file = here::here('materials', 'data', 'starwars_text.csv')) {
  
  read_csv(file)
  
}

# prepare data
clean_data = function(data) {
  
  data |>
    mutate(episode = case_when(document == 'a new hope' ~ 'iv',
                               document == 'the empire strikes back' ~ 'v',
                               document == 'return of the jedi' ~ 'vi')) |>
    mutate(character = case_when(character == 'BERU' ~ 'AUNT BERU',
                                 character == 'LURE' ~ 'LUKE',
                                 TRUE ~ character)) |>
    select(episode, everything())
}

# calculate sentiment
calculate_sentiment = function(data,
                               by = c("document", "character", "line_number")) {
  
  data |>
    sentiment_by(by = by) |>
    sentimentr::uncombine()
}

# define targets
list(
  tar_target(
    name = starwars,
    command = 
      load_data() |>
      clean_data()
  ),
  tar_target(
    name = sentences,
    command = 
      starwars |>
      get_sentences()
  ),
  tar_target(
    name = sentiment,
    command = 
      sentences |>
      calculate_sentiment()
  )
)

#> [[1]]
#> <tar_stem> 
#>   name: starwars 
#>   description:  
#>   command:
#>     clean_data(load_data()) 
#>   format: rds 
#>   repository: local 
#>   iteration method: vector 
#>   error mode: stop 
#>   memory mode: persistent 
#>   storage mode: main 
#>   retrieval mode: main 
#>   deployment mode: worker 
#>   priority: 0 
#>   resources:
#>     list() 
#>   cue:
#>     mode: thorough
#>     command: TRUE
#>     depend: TRUE
#>     format: TRUE
#>     repository: TRUE
#>     iteration: TRUE
#>     file: TRUE
#>     seed: TRUE 
#>   packages:
#>     readr
#>     dplyr
#>     sentimentr
#>     here 
#>   library:
#>     NULL
#> [[2]]
#> <tar_stem> 
#>   name: sentences 
#>   description:  
#>   command:
#>     get_sentences(starwars) 
#>   format: rds 
#>   repository: local 
#>   iteration method: vector 
#>   error mode: stop 
#>   memory mode: persistent 
#>   storage mode: main 
#>   retrieval mode: main 
#>   deployment mode: worker 
#>   priority: 0 
#>   resources:
#>     list() 
#>   cue:
#>     mode: thorough
#>     command: TRUE
#>     depend: TRUE
#>     format: TRUE
#>     repository: TRUE
#>     iteration: TRUE
#>     file: TRUE
#>     seed: TRUE 
#>   packages:
#>     readr
#>     dplyr
#>     sentimentr
#>     here 
#>   library:
#>     NULL
#> [[3]]
#> <tar_stem> 
#>   name: sentiment 
#>   description:  
#>   command:
#>     calculate_sentiment(sentences) 
#>   format: rds 
#>   repository: local 
#>   iteration method: vector 
#>   error mode: stop 
#>   memory mode: persistent 
#>   storage mode: main 
#>   retrieval mode: main 
#>   deployment mode: worker 
#>   priority: 0 
#>   resources:
#>     list() 
#>   cue:
#>     mode: thorough
#>     command: TRUE
#>     depend: TRUE
#>     format: TRUE
#>     repository: TRUE
#>     iteration: TRUE
#>     file: TRUE
#>     seed: TRUE 
#>   packages:
#>     readr
#>     dplyr
#>     sentimentr
#>     here 
#>   library:
#>     NULL

We can view the steps that will be carried out by pipeline using tar_manifest()

#>        name                        command
#> 1  starwars        clean_data(load_data())
#> 2 sentences        get_sentences(starwars)
#> 3 sentiment calculate_sentiment(sentences)

tar_visnetwork() provides a more detailed breakdown of the pipeline, including the status of individual targets, as well as the functions and where they are used.

We then run the pipeline using tar_make(), which will detail the steps that are being carried out and whether they were re-run or skipped.

targets::tar_make()

#> v skipped target starwars
#> v skipped target sentences
#> v skipped target sentiment
#> v skipped pipeline [0.049 seconds]

We can then load the objects using tar_read() or tar_load().

tar_load(sentiment)

sentiment |>
  head(10) |>
  gt::gt()

This might seem like a lot of overhead for little gain; if re-running is relatively painless, then is the it worth the time to set up a pipeline?

Embracing Functions

targets expects users to adopt a function-oriented style of programming. User-defined R functions are essential to express the complexities of data generation, analysis, and reporting.

https://books.ropensci.org/targets/functions.html

Traditional data analysis projects consist of imperative scripts, often with with numeric prefixes.

01-data.R
02-model.R
03-plot.R

To run the project, the user runs each of the scripts in order.

source("01-data.R")
source("02-model.R")
source("03-plot.R")

As we’ve previously discussed, this type of approach inherently creates problems with dependencies and trying to figure out which pieces need to be rerun.

For example, rather than write a script that loads, cleans, and outputs the Star Wars data, I simply wrote two functions, which we can easily call and run as needed to get the data.

Embracing functions makes it easier for us to track dependencies, explain our work, and build in small pieces that can be tested and put together to complete the larger project.

targets demo - College Football and Elo Ratings

• View repo organization for https:://github.com/ds-workshop/cfb_elo
• Examine targets.R
• Show _targets metadata
• Show _targets objects

Let’s revisit some of the original motivations for this workshop.

How do I share my code with you, so that you can run my code, make changes, and let me know what you’ve changed?

Let’s talk about using targets to build predictive models.